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The  "Utilization  Reviews"  required  under  the  Bennett  Amendment  to  the  Social 
Security  Act  of  1972  can  be  interpreted  as  "independent  audits"  of  the  prac- 
tices of  medical  professionals  utilizing  hospital  services.  This  includes  ai 
of  the  hospital  by  an  independent  Professional  Standards  Review  Organization 
Implementation  of  this  law  was  phased  in  a way  that  makes  it  possible  to  dis 
tinguish  between  "audit  anticipation"  and  "audit  occurrence"  as  separate  as- 
pects of  "audit  perse"  --  i.e.,  the  "audit  process"  as  distinguished  from  a 
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Cholecystectomy  from  a Boston  area  hospital,  techniques  from  Exploratory  Data 
Analysis  are  exploited  to  form  hypotheses  about  the  effects  of  these  audits. 
The  resulting  hypotheses  all  suggest  that  the  Utilization  Reviews  had  signlfii 
effects  on  the  behavior  of  medical  professionals  with  respect  to  length  of 
patient  stay  — which  were  then  confirmed  by  subsequent  statistical  (confirm- 
atory) tests  of  the  usual  (classical)  variety.  Openings  provided  for  further 
research  on  the  use  of  audits  as  a means  for  managing  the  behavior  of  such 
personnel  as  medical  professionals,  research  scientists,  etc.,  are  briefly 
indicated  along  with  other  new  management  tools  that  can  be  provided  from 
such  research. 
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ABSTRACT 


The  "Utilization  Reviews"  required  under  the  Bennett  Amendment 
to  the  Social  Security  Act  of  1972  can  be  interpreted  as  "independent  audits" 
of  the  practices  of  medical  professionals  utilizing  hospital  services.  This 
includes  audit  >f  patient-doctor  relations  by  staff  of  an  admitting  hospital 
as  well  as  audit  of  the  hospital  by  an  independent  Professional  Standards 
Review  Organization.  Implementation  of  this  law  was  phased  in  a way  that 
makes  it  possible  to  distinguish  between  "audit  anticipation"  and  "audit 
occurrence"  as  separate  aspects  of  "audit  per  se"  — i.e.,  the  "audit 
process"  as  distinguished  from  any  reports  flowing  from  it.  Using  data 
on  (1)  Appendicitis  and  (2)  Primary  Cholecystectomy  from  a Boston  area 
hospital,  techniques  from  Exploratory  Data  Analysis  are  exploited  to  form 
hypotheses  about  the  effects  of  these  audits.  The  resulting  hypotheses 
all  suggest  that  the  Utilization  Reviews  had  significant  effects  on  the 
behavior  of  medical  professionals  with  respect  to  length  of  patient  stay  — 
which  were  then  confirmed  by  subsequent  statistical  (confirmatory)  tests 
of  the  usual  (classical)  variety.  Openings  provided  for  further  research 
on  the  use  of  audits  as  a means  for  managing  the  behavior  of  such  per- 
sonnel as  medical  professionals,  research  scientists,  etc.,  are  briefly 
Indicated  along  with  other  new  management  tools  that  can  be  provided  from 
such  research. 
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Types  of  Audits 


This  article  will  explore  the  potential  value  of  audits  as  a 
tool  for  management  and/or  social  control  to  guide  the  behavior  of  medical 
professionals  in  a hospital  administration  context.  Control,  direction 
and  even  influencing  the  behavior  of  such  professionals  may  be  thought  of 
as  only  one  of  many  areas  which  have  provided  a variety  of  sometimes 
baffling  challenges  to  more  customary  management  approaches  — such  as 
direct  supervision,  etc.  — if  only  because  of  the  knowledge  discrepancies 
of  managers.  For  instance,  in  addition  to  medical  professionals  (e.g. , 
surgeons)  in  hospitals,  one  may  cite  the  activities  of  scientists  partici- 
pating in  program  of  advanced  research  or  even  the  activities  of  academics 
in  institutions  of  higher  learnings.  Ambiguities  in  locating  and  stating 
objectives  for  the  latter,  however,  make  it  seem  advisable  to  direct 
attertion  to  the  behavior  of  medical  professionals.  Here  the  ae  ambiguities 
are  less  intense,  e.g.,  in  their  bearing  on  patient  treatment,  etc.,  and 
then,  of  course,  what  is  learned  about  audit  effects  here  relative  to  these 
objectives  may  provide  a basis  for  extension  to  some  of  the  other  areas 
noted  above. 


The  term  "audit"  is  intended  here  to  refer  to  an  independent 
process  of  review  and  appraisal  that  carries  with  it  a third-party  orienta- 
tion (e.g.,  as  embodied  in  an  audit  report)  along  with  criteria  of  objectivity 
and  competence  that  makes  it  possible  to  appraise  the  audit  process  Itself 
as  well  as  the  activities  that  are  audited.  It  is  not  intended  to  confine 
the  term  "audit"  to  the  usual  CPA  attest  fid it  although,  to  bo  sure,  that 
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kind  of  audit  fit1',  within  the  description  we  have  just  provided.  We 


use  the  term  rather  in  the  sense  of  "comprehensive  audit"  which,  as  we 

have  elsewhere  noted,— ^ does  not  confine  an  auditor  to  examination  of 

financial  statements  prepared  by  others.  Instead,  it  opens  all  aspects 

of  management  (ai.  I other)  behavior  for  such  examination  and  assigns  the 

audit  or  responsibility  for  choosing  the  area  of  audit  which  is 

supposed  to  be  done  by  reference  to  the  3rd-party  relations  that  are 

2/ 

thereby  to  be  serviced.— 

In  comprehensive  audits,  the  choices  of  the  3rd-party  relations 


to  be  serviced  and  the  way  they  are  to  be  serviced  is  also  a responsi- 
bility of  the  auditor.  Note,  however,  that  this  servicing  need  not  involve  a 
formal  report.  Indeed,  for  purposes  of  this  study,  we  follow  Churchill 
[ 1962  ) anu  distinguish  between  "audit  per  so"  and  the  "audit  report." 

The  latter  is  further  distinguished  by  reference  to  the  "audit  findings" 
and  "audit  recommendations"  that  it  may  contain,  but  audit  per  se  refers 
to  the  audit  process  (i.e.,  the  scope  and  method  of  audit  conduct) 
whether  or  not  a report  issues  from  it. 

We  also  follow  Churchill  [ 1962  ] and  further  distinguish  between 
the  "occurrence  of  an  audit"  and  the  "anticipation  of  an  audit,"  which  may 
each  have  behavioral  consequences  in  their  own  right.  Quite  clearly,  no 


— ^See  N.  C.  Churchill,  W.  W.  Cooper,  V.  Govindarajan,  J.  D.  Pond,  and 
J.  G.  San  Miguel  [1977]  . 

2/ 

—For  further  discussion  of  this  servicing  of  3rd-party  relations,  see 
W.  W.  Cooper  and  Y.  Ijiri  "Accounting  and  Accountability  Relations"  in 
Eric  Louis  Kohler  — Accounting’s  Man  of  Principles  (Reston  Publishing 
Co.,  forthcoming).  For  a discussion  of  its  relation  to  accounting,  see 
the  exchange  between  A.  Thonas,  Mildred  ’’’■oncis  am'  V Govindarajan  and 
J.  Pond  in  "Evaluation  of  Social  Programs:  Some  thoughts  ci.  the  Role  of  Ac- 
countants," The  Accountin’  Journal,  Vol.  1,  No.  2 (1978). 
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report  is  involved  in  audit  anticipation,  at  least  in  the  sense  of  specific 
audit  findings  and/or  recommendations.  Indeed,  explicitly  specified 
audit  criteria  are  also  not  present.  Even  a clear  delineation  of 
the  actions  to  be  examined  need  not  be  present  unless  some  pre- 
arranged specification  of  procedures  to  be  followed  or  the  reports  to  be 

rendered  are  sufficiently  associated  with  the  possible 

1/ 

use  of  an  audit.- 


Audits  and  Utilization  Reviews 

For  purposes  of  this  study,  in  any  case,  the  3rd-party 
orientation  is  assumed  to  be  present,  but  with  possible  different 
behaviora]  effects,  in  both  audit  anticipation  and  audit  occurrence. 

To  test  hypothesized  behavioral  effects  of  both  of  these  aspects  of 
auditing,  recourse  was  had  to  data  on  the  possible  effects  of  Utiliza 
tion  Revit  vs"  both  in  anticipation  and  occurrence  in  a not-for- 
profit  teaching  hospital  in  the  Boston  area.  Hence  we  need  to  say 
something  about  the  nature  of  these  Utilization  Reviews,  including 
their  sources  and  authority,  and  tne  audit  processes  to  which  they 
relate. 

First  we  observe  that  the  Bennett  Amendment  to  the  focial  Security 
Act,  ' rom  which  t he  authority  stem;  lor  these  audits,  was  passed  in  October  30,  1972. 
As  observed  In  the  appendix,  this  amendment  provided  for  a PS  10  (Professional 
Standard  Review  Organization)  in  each  region  with  independent  authority 

— ^This  is  the  point  of  viev  that  •'«-  taken  In  che  present  study  where 
such  developments  of  recon  -keeping  and  reporting  practices  are  all 
legarded  as  flowing  from  the  audics  being  utilized. 
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for  establishing  standards  and  responsibility  for  monitoring  (i.e., 
externally  auditing)  the  Utilization  Reviews  and  related  audit 
activities  in  each  subject  hospital.  A grace  period  until  April 
1975,  contained  in  this  Amendment,  made  it  possible  to  study  the 
effects  of  these  audits  (-  Utilization  Reviews  + PSRO  monitoring) 
in  their  actual  occurrence  as  well  as  the  related  audit  ancicipa- 
tions  by  reference  to  data  collected  for  each  of  the  following  3 
periods  of  time: 


Prior  to  1972:  no  concurrent  audit  and  no 
anticipation  of  Utilization 
Reviews 

Nov.  1972  and  March  1975:  no  audit  occurrence  bu£  anticipation 

of  concurrent  Utilization  Reviews 
April  1975  - present:  concurrent  Utilization  Reviews  by 

the  hospital  staff  plus  external 
monitoring  by  the  loca]  PSRO 

This  provided  a "quasi  experimental"  setting  in  the  sense  of  Campbell 
end  Sianley  [ 1963]  which  was  utilized  to  analyze  the  effects  of  "audit  anti- 
, cipation"  and  "audit  oo-currence"  on  the  length  of  stay. 

• ^ Inter  alia  these  Utilization  Reviews  provide  a control  over 

attending  physicians  by  reference  to  criteria  established  by  the  PSRO 
with  respect  to  type  and  number  of  diagnoses  as  well  as  demographic 
and  other  characteristics  that  bear  on  the  nature  of  the  medical  treat- 
ment. These  criteria  are  intended  for  application  to  Issues  like  the 


BP{  it,  . 
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(extemal  audit)  monitoring,  they  also  provide  the  key  ingredients 
of  independence  and  the  third-party  orientations  that  we  earlier 
identified  with  audits.  Here  the  third-party  orientation  takes  the 
form  of  a peer  review  provided  by  other  medical  professionals  — 
as  well  as  others  — both  internally  in  the  hospital  and  in  the 
PSRO.  It  is  also  of  interest  to  observe  that  it  is  relatively  rare 
for  the  r<  suits  of  these  audits  to  be  reported  back  to  the  attending 
physician  in  the  form  of  a report  of  patterns  of  findings  and/or 
recommend  t ions.  That  is,  apart  from  the  control  process  that  is 


applicable  to  each  case,  there  is  no  audit  report  which  identifies 
entire  pat  terns  of  behavior  even  for  individual  physicians  as  a bails 
for  findings  and/or  recommendations.  Nevertheless,  as  we  shall  see, 
both  audit  occurrence,  in  the  form  of  audit  per  se,  and  audit  antici- 
pation ha  e significant  effects  on  patient  length  of  stay. 


Case-Mix  and  Tvpe  of  Variables 


One  oi  the  problems  encountered  in  some  of  the  earlier 
. g. , have  and  Leinhardt  [19761  ),  which 


studies 


arrived  at  conclusions  that  differ  from  ours,  is  that  they  madt  before/ 
after  comparisons  for  the  effects  of  Utilization  Reviews  with  respect 
only  to  the  average  length  of  stay  over  all  diagnoses.  Since  different 
diagnoses  with  varying  degrees  of  complications  generally  requite  dif- 
ferent lengths  of  hospital  stay,  such  aggregate  comparisons  could  be 
quite  misleading  especially  if  the  case-mix  varies  as  it  is  lxkely  to  do 
from  one  tim*  period  to  another  and  even  from  one  hospital  to  another.  One 
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way  of  eliminating  changes  in  case  mix  is  to  examine  the  length  of 
stay  for  each  diagnosis  separately.  We  have  made  such  separate 
comparisons  for  two  diagnoses,  and,  in  addition,  conducted  ex- 
tensive field  chocks  by  reference  to  interviews  (and  other  data) 
with  knowledgable  medical  professionals  not  only  in  the  instant 
hospital  but  in  others  as  well.— ^ 

Acute  appendicitis  and  primary  cholecystectomy  were 
chosen  for  this  purpose.  These  two  diagnoses  have  the  following 
characteristics  which  bear  on  our  analyses  and  conclusions: 

(a)  These  two  are  among  the  most  frequently 
treated  diagnoses  in  the  hospital.  They 
therefore  provide  a rich  data  base  to  work 
w i th . 

(b)  Patients  admitted  for  these  two  diagnoses 


d>>  not  develop  complications  very  frequently. 
Since  such  complications  not  only  increase 


the  length  of  patient  stay  but  also  require 
more  involved  analysis,  we  would  like  to  be 
able  to  deal  with  them  separately,  without 
thereby  greatly  diminishing  or  otherwise 

>. 

c 

weakening  our  data  base. 

(c)  There  have  been  no  substantial  technological 
advances  in  the  selected  diagnoses  during  the 
study  period  and,  further,  the  technological 

Cr  ^inda’-ajan  ( 1978  j f.  c the  results  of  these  interviews. 
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advancements  are  easily  traceable.  This  helps 
to  control  for  tins  influence  of  varying  medical 
technology  on  the  length  of  stay. 

Data  Collection:  For  both  diagnoses,  data  were  collected  for  each 
individual  treatment  on  the  following  variables  during  a nine-year 
period  from  July  1968  to  March  1977: 

1.  Code  number  of  the  attending  physician. 

2.  Code  number  of  the  attending  surgeon. 

3.  Complications  in  treatment. 

4.  Age  of  the  patient. 

5.  Sex  of  the  patient. 

6.  Month  in  which  patient  was  admitted  to  the  hospi'al. 

7.  Total  length  of  stay  of  the  patient. 

Exploratory  am  1 Confirmatory  Data  Analyses 

The  data  were  first  analyzed  using  certain  new  stati  tical 
techniques,  cal  Led  "Exploratory  Data  Analysis,"  EDA,  which  emphasize 
(a)  robustness  and  (b)  detection  and  analysis  of  outliers  [Tukey, 

(1977)].  In  EDA,  the  emphasis  is  on  median  and  median-related  tech- 
niques which  are  very  little  affected  by  stray  values.  That  is,  the 
emphasis  is  on  "robustness"  and  the  identification  of  "outliers." 

This  contrasts  with  more  customary  statistical  analyses, 

and  inference  procedures,  SAI,  which  emphasize  "sensitivity"  and  "central 
tendency. " 

There  is  no  necessary  cincLict  between  SAI  — witn  its 
emphasis  on  significance  tests,  ecc.  — and  EDA.  with  ita  etuphaais  on 


IS. 
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separation  of  outlier  and  nonoutlier  information  and  the  detection  of 
patterns  in  each.  Indeed  the  two  may  be  used  in  sequences  in  a very 
natural  strategy  wherein,  at  one  stage,  EDA  is  used  for  hypothesis 
development  and ^ at  another  stage,  the  well-known  procedures  of  SAI 
are  employed  to  test  these  hypotheses,  effect  estimates  and  determine 
quantitative  relations  which  may  be  used  to  effect  interpolations  and 
extrapolations  from  the  original  data  set. 

Because  EDA  is  at  present  relatively  new  (and  still  in  its 
early  stages) , we  shall  emphasize  its  use  in  the  discussion  that  fol- 
lows. Suffice  it  to  say  here,  therefore,  that  the  resulting  hypotheses 
have  now  been  confirmed  at  the  1%  level  of  significance  or  better  in 
almost  all  cases,  and  have  also  been  successfully  submitted  to  other 
tests,  too,  both  singly  and  in  combinations.—^ 

Before  proceeding  to  these  explanations,  however,  we  should 
sunmarize  the  result'2  of  our  study  to  date  as  follows: 

1.  The  evidence  indicates  that  borh  audit  anticipation 
and  audit  occurrence  have  pronounced  < ffects  in  the 
form  of  reduced  length  of  stay.  The  magnitude  o! 
audit  occurrence  is  significantly  greater  than  the 
magnitude  of  audit  anticipation. 

2.  These  reduced  length  of  stay  phenomena  appear  in 
both  outlier  and  nonoutlier  data. 

— ^E.g.,  via  customary  least  squares  regre3sion<  to  control  and  allow 
for  the  separate  and  combined  effect  of  /ariables  other  than  audit 
anticipation  and  occurrence  on  length  of  stay. 
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3.  The  nonoutlier  data  show  reduced  scatter 
(i.e.,  Increased  homogeneity)  while  the 
outlier  data  do  not  over  the  Indicated 
study  interval. 


The  above  summary  is  rough  and  subject  to  qualifications  (whicli  we 
shall  enter  later)  with  respect  to  parts  of  the  study  that  are  still 
under  way.  Nevertheless,  this  summary  will  help  in  focussing  atten- 
tion during  the  discussion  that  follows  on  the  use  of  EDA  as  one  stage 
of  a total  process  en  route  to  these  results.  Finally,  we  should  observe 
that  these  study  results,  which  involve  both  EDA  and  SAI,  have  produced  a 
variety  of  new  tools  such  as  regressions  (which  are  also  significant  sta- 
tist 5 call v) , and  which  can  bo  used  to  improve  the  processes  of 
hospital  i i.inageim-nr . This,  too,  could  be  counted  as  a further  con- 
sequence vf  the  audit/Utilization  Review  process  in  case  these  tools 
should  be  adopt 'd  by  this  hospital. 


Major  Findings:  Aggregate  Audit  Effects 
Before  proceeding  to  analyze  the  audit  effects  at  the  aggregate  hospital 
level,  we  first  attempted  to  identify  exogenous  factors  unrelated  to  the  Utilization  Re- 
views that  might  influence  patients’  stay  in  the  hospital.  Thi3  could  then  help  us  to  iso- 
i late  the  influence  of  these  exogenous  factors  and  direct  our  examination  of  length  of  stay 
for  possible  audit  effects  remaining  after  allowing  for  these  other  factors. 

Impact  of  Occupancy  Rates:  Occupancy  rate*  which  is  an  indication 
of  capacity  utilization  has  an  important  influence  on  the  patients'  length 
of  stay  in  hospitals.  If  the  occurcncy  »a»e  in  a no^piial  *s  very  high,  say 


Occupany  rate 


Number  of  in-pat ients  receiving  care  on  an  average  day/year 
(excluding  newborns! ______________________ 

Average  number  of  beds  per  d«*y  regularly  maintained  for  in- 
patients during  * year 


-U 


ma 


* mil  . . i M:i  ' % I: 
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close  to  100%,  then  doctors  might  ho  under  pressure  to  discharge  their 


patients  duo  to  shortages  of  beds  wh 

i eh , i n 

turn. 

tends  to 

shorten 

the  average 

length  of  hospital  dav.  To  examine' 

whether  such 

effects 

might  he 

present 

we  collected  data  on  the  occupancy 

rates  i 

n t he 

hosp  I tal 

weh  have 

chosen 

over  the  last  five  years*  which  we 

repor L 

as  follows:** 

1 72 

'73 

'74 

'll 

'76 

'77 

Occupancy  Rite  79.2% 

78.6% 

81.2% 

77.5% 

78.3% 

75.6% 

Patient  Day.  83,462  87,291 

94,173 

95,918 

97,142 

93,514 

The  o>  up.,  cy  rates  In  the  last  5 yea*, have  been  below  80%  and  further  the 
occupancy  i ate  is  steadily  falling  over  the  5 year  period.  These  observations 
indicate  that  there  is  still  slack  in  the  availability  of  hospital  beds,  i.e., 
the  doctor:  tro  not  under  pressure  to  discharge  their  patients  < ue  to  shortage 
of  hospita  beds.  Thus  it  is  not  plausible  to  attribute  reductions,  if  any, 
in  the  ave  age  Length  ot  hospital  stay  in  our  hospital  to  pressure  emanating 
from  high  nd/or  > ising  occupancy  rates. 

mpact  of  Age,  Sex  and  Complications  in  Treatment:  Exhibit  1 gives  what 
is  called  in  EDA  a stem-and-leaf  display  of  the  length  of  stay  on  al 1 the  778  patients 
treated  foi  acute  appendicitis  during  1968-77.  In  the  display  inc  luded  in  Exhibit  1 
each  of  th<  digits  2 to  11  on  the  left  of  the  veritcal  line  is  a stem,  and 

^Strictly  speaking,  since  we  are  dealing  with  acute  appendicitis,  an  examination 
of  the  hospital-wide  occupancy  rates  is  appropriate  only  if  beds  are  perfectly  sub- 
stitutable among  different  departments.  The  difference  appears  moot,  however,  since, 
informal  discussions  with  hospital  officials  revealed  that  slack  bed  capacity  exists 
oven  in  the  departenent  which  handles  the  two  diagnoses  we  have  chosen  for  study. 

**Total  patient  days  in  1974  took  a sharp  increase  due  to  21  new  beds  added 
in  psychiatry  in  December  1973.  These  21  beds  had  an  occupancy  rate  of 
92%  during  1974. 


JL 
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each  remaining  digit  placed  to  the  right  of  the  vertical  line  is  part  of  a 
leaf  (in  our  case,  these  are  all  equal  to  zero). 

The  display  in  Exhibit  1 can  be  understood  better  by 
comparing  it  to  the  familiar  histogram  which  is  constructed  using 
class  intervals.  Each  utem  in  the  above  display  has  a class  interval 
of  1 day.  The  leafs  corresponding  to  each  stem  give  the  number  of 
data  values  in  that  stem.  This  idea  is  similar  to  a histogram  where 
the  Length  of  each  bar  is  proportional  to  the  number  of  data  values 
in  that  interval  (often  called  "frequency").  However,  a stem-nnd-leaf 
display  has  an  advantage  over  a histogram,  in  that  the  former  gives 
numerical  detail  in  their  leaves  whereas  the  length  of  each  bar  in  a 
histogram  is  either  a blank  or  a shaded  space  as  if  all  data  w re 
"centered"  in  each  class  interval.*  Note  that  it  is  therefore  pos- 
sible to  go  back  from  what  appears  in  a stem-«»nd-leaf  display  to  the 
full  "raw"  data  val'-es. 

One  advantage  of  the  stem-and-leaf  display  as  an  aid  to  hypo- 
thesis formulation  — for  subsequent  confirmatory  statistical  analyses  — 
is  that  ii  helps  (or  even  forces)  us  to  see  both  the  regularities  and 
Irregularities  In  a batch  of  data  values.  For  instance,  we  can  see 
from  Exhibit  1 that  most  patients  had  stayed  between  4 and  8 days. 

At  this  stage,  however,  we  were  particularly  interested  in  iden- 
tifying possible  "outliers"  in  the  patient  days  data,  so  that  we  could 
give  them  special  attention.  For  this  purpose,  we  computed  a "five  number 
summary"  for  the  data  and  the  results  are  as  follows: 

£ 

Thie  contrast  could  be  brought  out  more  sharply  if  our  dace  had  permitted 
us  to  regard  "fractions  of  a day"  in  place  of  the  0's  (whole  day  units) 
in  our  leafs. 
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2 (4,  S,  7)  119* 

where  2 ■ Extreme  Value  (Lowest) 

4 - Hinge  (Lower) 

5 ■ Mt  dian 

f * H ige  (Upper) 

11>|  m E treme  Value  (Highest) 

Here  wo  have  agai  i used  the  terminology  of  Exploratory  Data  Analysis.  These  5 
values,  it  ni  iy  be  observed  are  fpr  variables  which  are  approximations  to  well  known 
statistical  concepts.  For  instance,  the  "First  Quarter"  and  "Third  Quarter"  are 
the  same  as  the  corresponding  "quartiles"  in  which  we,  following  Tukey,  replace  the 
latter  terms  with  the  former  in  the  interest  of  simplicity  of  expression.  In  the 
classical  literature,  the  difference  between  the  maximum  and  minimum  values  gives 
the  "range,"  =119  - 2,  and  the  difference  between  the  third  quartiJe  and  the  first 
quartile  gives  the  "interquartile  range."  Tukey  refers  to  the  difference  between 
the  lower  a.  d upper  hinges  as  the  Q-spread  = 7 - 4 , again  to  provide  an  easiness  in 
expression  I ut  which  again  is  an  approximation  to  the  classical  interquartile  range . 

The  above  summary  gives  several  useful  insights  into  a batch  of  data  Wi 
can  get  a I < el  ’or  the  extreme  values  (in  our  case,  the  lowest  and  highest  values 
are  2 days  end  119  days,  respectively)  as  well  as  the  most  typical  value  in  the  batch 
(in  our  case,  the  median  is  5 days).  Further,  the  Q-spread  gives  us  a useful  indica- 
tion of  a middle  amount  of  spread  for  the  batch  (in  our  case,  the  Q-spread  is  7-4.  = 

3 days).  Note  that  uni  ike  the  variance  or  standard  deviation,  the  Q-spread  does  not 

depend  on  underlying  assumptions  of  symmetry  for  its  interpretations.  Indeed,  the  Q- 
spread  can  help  us  to  identify  properties  such  as  skewness  in  the  data  when  we  relate 
it  to  other  measures  such  as  the  median,  and  this  is  what  we  want  at  this  stage  of  an 
analysis. 

*The  depth  of  a data  value  is  the  number  ef  data  values  one  oust  count  in  from 
the  nearer  und  of  the  batch  n orccr  to  r »ch  the  pa»:ticul«i  data  value.  E~ch 
extreme  value  — the  lowest  <ad  the  highest  — is  at  depth  1.  *he  median  is 
at  depth  (1  + tota*  number  of  observations) /2  , i.e.,  median  car.  be  found  by 

counting  half-way  from  one  extreme  tc  the  other.  The  two  hinges  — upper  end 

lower  — can  be  found  by  counting  half-way  fre-o  each  extreme  to  the  median, 
f.e.,  the  depth  of  each  hinge  is  ■ + (depth  of  median! 
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The  not  Ion  of  Q-spread  can  further  help  us  detect  stray  values 
or  outlieis.  Tukey's  rule  for  detecting  outliers  Is  based  on  these 
definitions: 

Step  : 1.5  x Q-spread 

Fence  : 1 step  outside  each  hinge 

A value  beyond  the  (upper  or  lower)  fence  Is  tagged  as  an  outlier. 

Using  this  rule,  we  identified  the  38  outlier  values 
indicated  at  the  bottom  of  Exhibit  1.  We  then  proceeded  to  examine  these 

18  cases  individually  to  see  whether  they  had  any  characteristics  that 

distinguished  them  from  the  rest  of  the  740  cases.  Out  of  the  38  cases,  (a) 

27  were  associated  with  medical  or  surgical  complications  or  both  and/or  (b) 

they  represented  patients  who  were  older,  i.e.,  above  65  years  of  age.  We 

therefore  hove  information  which  indicates  that  medical  and  surgical  complications 

ire  present  that  entail  a longer  treatment  for  thase  cases.  Also  older  patients  tcid  to 

develop  complications  jr.d  hence  would  require  a longer  hospital  stay. 

We  proceeded  to  study  this  further . The  correlation  matrix  given  in  Exhibit  2 
also  suggest.1;  the  already  indicated  relationship  between  medical  and  surgical  complications, 
on  the  one  hand,  and  length  of  stay,  on  the  other  (correlations  of  0.5092  and  0.3903,  respec- 
tively). See  also  the  irtercorrelation  between  Secondary  Diagnosis  and  Secondary  Surgery , 
which  is  0.4731.  Similarly,  age  appears  to  have  a relatively  high  relationship  with  medical 
complications  (correlation  of  0.3028)  and  in  turn,  with  hospital  length  of  stay  (correlation 
of  0.3224). 

To  explore  these  ideas  still  further,  we  cross-tabulat' d (using  med- 
ians) the  length  of  stay  witn  different  types  of  comp] ications,  different  age 
group  of  pa'jem.s  and  sex  c ' c ssif  ications.  Ti,e  results  of  these  cross- 
tabu la lions  are  given  in  the  following  Exhibits: 


i 
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Cros s - T abu latlons  Using  Medians 


Exhibit  No. 

Dependent  Variable 

Independent  Variable(s) 

3a 

Length  of  stay 

Secondary  Diagnosis 

3b 

ft 

Secondary  Surgery 

3c 

• 1 

Sex 

3d 

If 

Age 

3e 

II 

Secondary  Diagnosis  and 
Secondary  Surgery 

3f 

II 

Secondary  Diagnosis, 
Secondary  Surgery,  and 
Age 

The  following  points,  based  or.  an  analysis  of  the  above  Exhibits,  can  be 
highlighted : 

(1)  Medical  and  surgical  complications  do  have  an  important  effect 

on  the  patients'  hospital  stay.  That  is,  as  we  proceed  from 
no  complications  ("0"  level)  to  more  complicated  treatments 
("1",  "3",  etc.  levels),  median  length  of  stay  increases 

appreciably  (Exhibits  3a  and  3b). 

(2)  Sex  has  no  effect  on  hospital  stay  since  both  males  and  females 
have  an  identical  median  length  of  stay  of  5 days  (Exhibit  3c). 

This  is  expected  since  acute  appendicitis  is  not  supposed  to 

I 

result  in  longer  lengths  of  stay  for  males  as  opposed  to  females 
or  vice  versa. 


(3)  The  age  of  the  patient  has  a strong  influence  on  the  length  of 
stay.  The  median  length  of  stay  for  patients  who  are  65  years 
or  younger  is  5 days  whereas  patients  who  are  older  than  65  years 
have  a median  length  of  stay  of  12  days  (Exhibit  3d). 


> i 
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(4)  The  two-way  and  three-way  cross-tabulations  given  in  Exhibits  3a 
and  3f  also  point  up  to  the  same  conclusions  mentioned  in  (1)  and  (3) 

above . 

Note  that  we  have  used  a robust  (distribution  free,' statistic  in  the  form  of  the 
median  rather  than  the  more  sensitive  mean  (which  is  not  distribution  free)  so 
that  the  above  differences  indicate  the  presence  of  strong  unde’-lying  causal  re- 
lations for  the  indicated  variations.  In  any  case  our  analysis  so  far  has  shown 
that  only  the  hospital  stay  of  663  patients,  i .e. , patients  who  nave  no  complica- 
tions and  whose  age  is  65  years  or  less,  need  to  be  examined  to  identify  possible 
"normal"  (i.e.,  non-outlier)  effects  of  Concurrent  Utilization  Reviews. 

Impact  of  Long-Term  Trend:  Of  course,  the  static  analysis  such  as 

we  have  not  conducted  does  nut  cover  other  things  that  also  need  to  be  consid- 

ered. Issues  of  i rend  ticad  to  be  considered  and  also  variation:,  about  measu  es 
of  central  tendency  both  statically  and  over  periods  o(  lime. 

Exhibit  4 presents  a plot  of  the  mean  length  ol  stay  over 

105  months.  This  plot  indicates  a tendency  for  the  length  of  stay  to 
go  down  as  we  proceed  in  time,  and  we  are  interested  in  finding  out 
how  much  of  this  decrease  can  be  attributed  solely  to  a long-term  downward 
trend  in  the  length  of  stay.  It  is  possible  that  a gradual  improvement  in 
the  health  standards  of  the  population  over  the  years  might  contribute  to 
« gradual  lowering  of  the  length  of  stay  since  the  initial  medical  condition 
of  a patient  does  influence  the  patient's  stay  in  the  hospital.  Also, 
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technologic.nl  advances  could  have  brought  down  the  length  ot  stay  over  time  .* 

Our  interest  here  was  to  identify  the  long-term  trend,  if  any,  in  the  particular 
diagnosis  we  have  chosen  for  study,  i.e.,  acute  appendicitis.  Thus  we  confine  ourselves 
to  this  diagnosis  and  turn  to  the  first  of  these  possibilities  in  order  to  as- 
certain whether  an  identifiable  time  trend  is  present  in  the  data.  For 
this  put  pose  wo  utilized  the  data  over  51  months  from  July  1968  to  October  1972** 
to  estimate  the  following  equation  using  the  technique  of  robust  regression : 

Ak  A A 

- a + 6 T1  (1) 

A ^ 

where  Y^  = Length  of  stay  in  the  i month 

= Linear  time  variable  (monthly)*** 

The  numerical  results  are  as  follows: 

A 

Y - 7.000  - 0.000000003725 

Average  sum  of  the  absolute  residuals  - 1.534 

In  the  above  estimates,  we  notice  hardly  any  trend  in  the  data  since  the  Beta 
coefficient  corresponding  to  the  time  variable  is  almost  zero./1 


*Recall  that  we  earlier  eliminated  pressure  on  bed  space  as  a possible  source  of 
this  trend. 

**We  considered  this  period  since  there  was  no  Concurrent  Utilization  Reviews 
during  this  period  and  therefore,  no  confounding  influences  due  to  audit 
anticipation  or  audit  occurrence  will  be  present. 

***Since  we  have  no  £ priori  information  pointing  otherwise,  we  fitted  a 
linear  trend  line. 

#This  is  not  statistically  significant  but  even  if  it  were  statistically 
significant  the  coefficient  is  so  small  that  it  could  not  be  accorded 
either  administrative  or  economic  significance.  Note,  however,  that  any 
ordinary  (least  squares)  regression  is  with  respect  to  the  mean  whereas 
ne  robust  regression  is  formed  relative  to  the  median,  a much  less 
.ensitive  statistic.  Thus  a least  squares  (ordinary)  regression  might  hove 
given  <i  different  result  than  the  lormer. 
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We  had  estimated  the  trend  using  data  over  a fairly  long  period  of 
51  months  and  could  find  no  evidence  to  support  the  presence  of  any  long-term 
trend  for  this  particular  diagnosis.  It  should  perhaps  be  pointed  out  here  that 
since  our  emphasis  is  on  Exploratory  Data  Analysis,  isolating  the  exogenous  fac- 
tors one  at  a time,  is  justified  as  we  have  done. 

We  further  discussed  our  data  analyses  with  a few  surgeons  in  the  in- 
stant and  other  hospitals.  The  interviewees  also  could  not  point  up  any  possible 
influence  of  long-term  trend  including  technological  advances  on  the  length  of 
stay  for  acute  appendicitis.  Thus  we  provisionally  rule  out  such  trend  possi- 
bilities and  turn  to  other  factors. 

Analysis  for  Aggregate  Audit  effects 

First,  we  divide  our  time  periods  in  a different  w.iy  than  before  in 
order  10  examine  the  possible  effects  of  audit  anticipation  an  i audit  occurrence 
on  the  lengt  > of  stay.  We  will  do  so  by  separating  out  noise  :nsofar  as  we  can 
identify  it  now,  by,  for  instance,  considering  only  the  data  fnr  the  663  patients 
of  age  65  or  less  who  had  no  complications. 

We  divide  the  663  cases  identified  earlier  in  the  following  manner: 

(a)  310  were  treated  during  July  1968-October  1972  when  there  vas  neither  occur- 
rence of  Concurrent  Utilization  Reviews  nor  any  plausible  basis  to  believe  that 
anticipation  was  present  that  such  reviews  will  occur  (to  be  referred  to  as  "no 
audit"  period) ; (b)  220  patients  were  treated  during  November  1972-March  1975  when 
it  is  plausible  to  believe  there  was  anticipation  that  such  reviews  would  occur 
(to  be  referred  to  as  "audit  anticipation"  period) , and  (c)  133  patients  were 
treated  during  April  1975-March  1977  when  there  was  actual  occurrence  of  such 
reviews  (to  be  referred  to  as  "actual  audit"  period).  It  appears  from  the  plot 
in  Exhibit  4 that  the  median  length  of  stay  during  the  "audit  anticipation 
and  "actual  audit"  periods  are  lower  as  compared  to  the  "no  aun  It"  period. 
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More  importantly,  the  "no-nudlt"  period  exhibits  more  variance  than  the  othe^ 
two  periods. 

4 

To  further  examine  the  audit  effects,  we  prepared  stem-and-leaf 
displays  of  the  length  of  stay  during  the  three  periods  (Exhibits  5a,  5b, 
and  5c).  Tt  can  be  seen  from  these  Exhibits  that  length  of  hospitalization 
is  clustered  around  6 days  for  the  no  audit  period,  5 days  for  the  audit 
anticipation  period,  and  4 days  for  the  actual  audit  period.  his  can 
also  be  seen  by  examining  the  medians  in  the  following  5-number  summaries: 

No  Audit  Period  : 2 (5,  6 (Median),  8)  18 

Actual  Anticipation  Period  : 2 (4,  5 (Median) , 6)  17 

Actual  Audit  Period  : 2 (4,  4 (Median) , 5)  14 

It  aaems  evident  from  the  following  figures  that  both  the  median  hospital 
atay  as  well  as  the  spread  around  the  median  decline  as  we  proceed  from  (a) 
no  audit  lo  (b)  audli  anticipation  to  (c)  actual  audit: 

Medlar.  Length  of  Slay  Q-Spread 
No  Audit  Period  6 days  3 days 

Audit  Anticipation  Period  5 " 2 " 

Actual  Audit  Period  4 " 1 " 

"everting  to  another  technique  of  Exploratory  Data  Analyses  w » can  bring  out 

the  audit  effects  indicated  above  more  dramatically  b / means  of  the 

"box  plot."  for  the  length  of  stay  for  these  three  periods  which  is  displayed 
In  Bxhibl i 8.  We  proceed  to  explain  what  is  involved  as  follows:  We  can  il- 
lustrate the  technique  of  constructing  such  a plot  by  examining  the  anatomy  of 
the  following  schematic  plot  (similar  to  the  plot  in  Exhibit  6)  which,  as  ve 
can  see,  is  based  on  an  augmentation  of  the  5-nurber  summary  idea  discussed  earlier. 
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constuct,  but  effective,  visual  technique  for  comparing  related  batches  of  data 
or,  rather,  data  summaries.  By  constructing  such  schematic  plots  for  several  batches 
side  by  side,  we  can  gain  insights  by  reference,  e.g.,  to  (a)  the  level  of  the  me- 
dian in  the  various  batches,  (b)  the  amount  of  spread  in  each  batch,  and  (c)  the 
number  and  value  of  outliers  in  each  batch. 

Analysis  of  non-outliers:  From  Exhibit  6 we  note  t hat  the  median  value 
of  the  length  of  day  and  its  spreao  decrease  as  we  proceed  fr'm  no  audit  antici- 
pation to  actual  audit  period. 

Analysis  of  Outliers:  We  next;  identified  the  outli  ;rs  for  the  tbrae  p«rd- 
ods  which  are  as  follows: 


I 
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No  Audit  Period 

Audit-  Anticipation  Pei  led 

Actual  Audit  Period 


Out  lie  i s (in  days) 

13,  13,  14,  15.  18 

10,  12,  12,  IT.  17,  17 

7,  7,  7,  7,  8.  8,  8,  9,  10,  14 


A separate  analysis  of  the  outliers  indicated  above  shows  two  patterns: 
(a)  there  is  some  downward  trend  in  the  size  — not  only  in  median  but  also  in  the 
maximum  value  — of  the  outliers  over  the  three  period'..  This  may  indicate  that 
audit  in  its  occurrence  is  likely  to  discourage  too  much  deviation  from  the  audit 
criteria.  (b)  The  scatter  varies,  however,  and  in  fact  there  is  an  increasing 
scatter  in  the  outliers  over  the  three  periods.  This  nay  indicate  more  discrimina- 
tory treatment  during  the  audit  period  as  compared  to  'he  pre-audit  period.  These 
two  patterns  are  also  visible  in  the  dots  at  the  top  or  the  box  plot  given  in 


Exhibit  6. 


Results  for  Primary  Cholecystectomy:  Having  developed  our  results  for 


acute  appendicitis  in  some  detail  we  can  condense  our  discussion  for  the  affects 
of  Utilization  Review  in  the  case  of  Primary  Cholecystectomy.  We  might  note,  how- 
ever, that  similar  audit  effects  were  noticed  for  primary  rhole  ystectomy  based 
on  our  study  which  we  summarize  as  follows: 

We  analyzed  416  itncomp I icated  cases  of  primary  cholecystectomy  and 


observed  the  following  results: 


No  Audit  Period 

Audit  Anticipation  Period 

Actual  Audit  Period 


Mean  Length 
of  Stay 

11  days 
9 days 
8 days 


0-spread 


This  reduction  in  the  median  and  its  spread  is  brought  out  in  the  box  plot 
given  in  Exhibit  7. 

In  summary,  we  notice  the  following  audit  effects  for  both  diagnoses: 
(1)  For  the  bulk  of  the  observations,  audits  — both  anticipation 

and  actual  occurrence  — tend  to  bring  down  the  average  length 
of  stay  and  its  variance. 
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(2)  In  the  case  of  outliers,  audits  tend  to  decrease  the  size  of 
outliers  but  increase  var I ability  in  the  measures  of  scatter 
around  the  average  length  of  stay. 

Thus,  we  can  say  in  an  overall  way  that  audit  effects  were  uniformly  de- 
tected in  the  hypoi hesized  directions  with  actual  audit  consistently  having  a 

significantly  stronger  effect  than  audit  anticipation. 

Major  Findings:  Individual  Surgeon  Level 

So  far  we  have  analysed  audit  effects  only  at  the  aggregate  hospi- 
tal level.  We  will  now  focus  on  the  indiviudal  surgeons  to  see  v/hether  we  can  ob- 
serve similar  results  at  the  individual  surgeon  level.  Inter  alia  this  will  help 
us  to  deal  with  issues  such  as  possible  changes  in  the  mix  and  practices  of  the  sur- 
gical staff. 

Analysis  by  Surgeons:  Here  we  shall  only  consider  doctors  who  have 
treated  20  o more  c3ses  for  the  two  diagnoses  during  the  9 year  period  from  July 
1968  to  March  1977.  Such  a list  of  doctors  is  contained  in  Exhibits  8 and  9.  Given 
this  list  we  then  proceeded  to  make  before/after  comparisons  of  the  median  length  of 
stay  and  the  spread  around  tho  median  for  each  of  the  doctors  identified  in  Exhibits 
8 and  9.  The  results  are  attached  as  Exhibits  10  and  11  which  we  summarize  as  follows: 

(1)  For  both  diagnoses,  audits  — both  anticipation  find  actual  occur- 
rences — produced  effects  in  desired  directions  for  the  majority 
of  surgeons. 

(2)  Even  though  most  of  these  surgeons  responded  favorably  to  audits, 
chere  were  differences  in  that  (a)  the  audit  effects  were  greater  in 
the  case  of  some  surgeons  than  others  and  (b)  in  some  few  cases  the 
results  serried  reversed.  See  the  surgeons  coded  as  numbers  *> 
(Exhibition  10)  and  10  (Exnilition  11). 
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Implications 

We  will  now  consider  some  of  the  major  implications  of  our  results  on 
practice.  Hospital  administrators,  who  usually  have  only  a management  background 
can  experience  problems  in  terms  of  comprehending  and  controlling  the  activities 
of  surgeons  and  other  medical  practitioners  subject  to  their  management  authority. 
New  tools  of  management  seem  to  be  called  for  and  audits  seem  to  offer  one  such 
possibllty.  This  study  has  helped  to  confirm  the  potential  value  of  audits  for 
these  purposes. 

These,  we  may  observe,  are  peer  group  audits.  Although  they  depart 
from  CPA  attest  audits  of  management  financial  reports  in  various  way,  they  never- 
theless have  the  requisite  properties  of  independent  review  and  criteria  of  objec- 
tivity and  validity  to  provide  a basis  of  evaluation  for  the  audit  process  itself. 

In  our  studv,  we  focussed  on  "audit  anticipation"  and  "audit  occur- 
rence" as  parts  of  "audit  per  so."  These, too,  cou’d  be  taken  into  account  in  a 
suitable  arranged  strategy  ranging  from  audit  announceranet  through  audit  per  se  t<> 
audit  report  in  controlling  such  "dif f icult-to-manage"  areas  such  as  the  ones  pre- 
sent in  hospitals. 

Even  though  we  focussed  on  Utilization  Reviews,  our  results  should 
have  implications  for  other  types  of  audits  in  hospitals  (e.g. , tissue  culture 
reviews)  which  evaluate  quality  of  patient  care.  Indeed,  as  noted  in  the  opening 
paragraph  of  this  paper,  further  esearch  directed  to  other  difficult  to  manage 
areas  such  as  the  administration  of  t esearch  programs,  etc.  seems  indicated.  Mor< 
generally  our  results  also  indicate  new  possibilities  for  internal  audit  as  a tool 
of  management  in  private  and  public  sector  organizations  as  veil  as  using  external 
audit  for  the  evalu-.tion  of  sucn  organizations.  Strategies  based  er  mix'-s  of  audit 
anticipation  and  actual  audit  might  then  be  used  to  conserve  audir  resources  and 
Improve  managerial  behavior  as  a result  of  research  in  these  areas,  too. 
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APPENDIX  A 

RESTATEMENT  OF  THE  BENNETT  AMENDMENT  (P.1.92  603) 


The  purpose  of  I’.l  .‘•2-W).>  is  in  promote  the  elTeetive.  efficient,  and 
eeonomieal  delivery  ol  Medicare  and  Medicaid  health  servi  es  Ihrouoh  the 
application  ol  prolessional  slalidaids  icview  pincc.luics  | he  ptovision  is  de  .lulled 
to  assure  proper  utilization  ol  care  and  services  through  a formal  professional 
mechanism  representinj’.  the  htoailest  possible  eri'ss-seelion  of  pr:o  liciiig  physi- 
cians in  an  aica  I lie  review  would  a .sine,  within  rceoeni/.vd  professional  health 
care  standards,  that  medical  services  linaneed  under  the  Social  Sccuiily  Act  are: 


• Of  appropriate  quality, 

• Provided  only  when  necessary,  and 


Provided  in  the  most  economical  fashion. 


Professional  Standards  Review  Organizations 

The  Secretary  will  establish  appropriate  areas  throughout  the  country  by 
January  I,  1974.  and  at  the  earliest  piacticablc  date,  designate  a qualified 
organization  as  the  l’SRO  for  each  aiea. 

A qualified  organization  will  be  a nonprofit  professional  association,  with 
voluntary  membership,  iiieludmu  a substantial  munber  of  the  pr  .tiv.ir.’  physi- 
cians in  the  area  (gei.e rally  slated  at  tf)0  or  wore).  Membership  m the  I'SI'O  and 
service  for  the  l’SRO  cannot  be  constrained  hi  a requirement  lor  member  iiip  in 
or  payment  of  dues  to  a medical  society.  The  qualified  organization  must 
domonsliale  its  professional  eompelenee  lo  review  health  care  services  in  a lormal 
plan  submilled  to  the  Secretary. 

Until  January  I.  1976.  the  Seeietary  will  be  able  to  make  such  agreements 
only  w'ith  a qualified organization  which  represents  a substantial  proportion  of  the 
physicians  in  the  geographical  area  designated.  Until  January  I,  1976.  at  the 
request  of  10  percent  or  more  of  the  practicing  physicians  in  a geographical  area 
designated  by  the  Secretary,  the  Secretary  will  be  required  to  poll  the  practicing 
physicians  in  the  area  .is  to  whether  or  not  an  organization  of  physicians  which 
has  requested  to  conclude  an  agreement  with  the  Secretary  to  establish  a PSRO 
substantially  represents  the  practicing  physicians  in  that  area.  If  more  than  50 
percent  of  the  practicing  physicians  in  the  area  responding  to  the  poll  indicate 
that  the  organization  does  not  substantially  represent  the  practicing  physicians  in 
the  area,  the  Secretary  will  not  enter  into  an  agreement  with  that  organization. 
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After  January  I.  I‘>7f«  :.  any  area  wl  -to  Micii  .1  professional  organization  is 
unwilling  or  unable  to  meet  these  conditions.  the  Secretary  nuy  designate  any 
public,  nonprofit  private,  or  other  agency  m organization  as  the  PSRO  providing 
that: 

• The  final  dctctmin.uions  of  peei  review  are  nude  by  physicians,  and 

• 1 hat  the  agreement  will  not  be  renewed  if  a qualified  professional 
organization  is  willing  and  able  to  become  the  PSRO  for  the  area. 

Each  PSRO  will  review  the  utilization  and  quality  of  all  institutionally 
provided  services  in  the  designated  aiea  I he  PSRO  will  not  be  involved  with  the 
determination  of  reasonable  charges.  With  the  concurrence  of  the  Secretaiy.  the 
PSRO  may  also  review  other  than  institutional  setviees.  These  reviews  will 
determine  that: 

• The  services  and  items  are  or  were  medically  necessary, 

• The  quality  of  the  services  meet  professionally  recognized  standards  of 
health  care,  and 

• The  services  are  or  were  provided  at  the  most  economical  level  of  care 
appropriate  to  the  need  (i.c  , outpatient,  extended  care,  hospital,  etc. I 

To  accomplish  lit. ' • purposes,  the  PSRO  will  utilize  and  apply  r gion.J 
norms  of  care  and  treatment,  based  upon  typical  patterns  of  practice  in  the 
legion,  pu  pated  tc.l  rw-.ed  by  the  National  ('<>111101.  Norms  will  be  developed  m 
accoi J.i'icc  with  the  legnlatuuis  of  the  Sect,  laiv  and  will  in  lude: 

• ’flic  type  and  extent  of  service  within  the  range  of  appropriate 
treatment  for  each  illness  or  health  condition,  and 

• The  type  of  health  care  facility  which  can  most  economically  provide 
appropriate  medical  services  for  each  illness  or  health  condition. 

Each  PSRO  will  utilize  the  norms  described  above  as  the  standards  for 
evaluation  and  will  perform  the  following  (unctions: 

(If  Prospeclive  determination  of  the  medical  necessity  and  appropriateness 
for  elective  admissions  and  extended  or  costly  courses  of  treatment. 

(?)  Periodic  determination  and  publication  of  selected  types  and  kinds  of 
cases  (by  service,  diagnosis,  or  other  criteria)  for  intensive  and  more 
effective  review. 


r 
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(3)  Regular  review  of  profiles  of  cate  for  selected  patient  groups, 
practitioners,  and  other  piovidcro  of  service. 

(4)  Time  specification  for  the  certification  of  continuing  inpatient  care  not 
Intel  than  the  regional  norm  lor  the  'Oth  percentile  of  length  of  stay 
for  patients  in  similar  age  pumps  with  similar  diagnoses. 

(5)  Require  physician  certification.  and  supporting  justification  for  con- 
tinued inpatient  care  beyond  these  specified  points  in  lime. 

(6)  luster  the  acceptance  ol  I'SIU)  functions  by  broad  physician  participa- 
tion. totaling  memheiship  on  review  committees,  wide  specialty 
representation,  and  publication  of  I’SRO  activities  in  appropriate 
professional  publication. 

(7)  Report  any  vioiat.ui.s  of  the  obligations  of  a practitioner,  hospital,  or 
other  health  care  ta.ilitv.  agency,  or  organization,  under  this  Program, 
to  the  State  (Vun-.ii  including  any  pertinent  recommendations. 

(8)  Notify  any  pmctitionci  or  provider,  and  provide  opportunity  for 
discussion  and  review,  when  any  deteunination  denies  a request  for 
services  or  identities  a violation  of  any  obligation  by  the  practitioners 
or  provider. 

(9)  Notify  the  pertinent  intermediary  of  any  disapproved  services  or  items. 

Physicians  ass'-m-J  the  responsibility  for  hospital  review  should  have 
hospital  privileges  m •!  ! ,i> t one  of  the  participating  hospitals  'll  the  area  and 
should  not  be  rcM  ote  > Ic  for.  but  may  p.nticipale  hi.  the  review  of  services  in 
their  own  hospital  \'>  physician  will  review  services  in  which  he  was  directly  or 
indirectly  involved  or  scivices  piovided  I \ any  institution,  organization,  or  agency 
in  which  he  or  his  handy  li.e  e a vested  interest. 

After  notification,  any  beneficiary  or  recipient  who  is  dissatisfied  with  a 
decision  made  with  r.;  ard  to  his  claim,  will  be  entitled  to  a reconsideration  by  the 
PSRO.  This  decision  may  be  appealed  to  the  State  C ouncil,  if  the  determination 
involves  more  than  S 100.  Where  the  decision  of  the  State  Council  is  adverse,  ot  in 
the  absence  of  a State  Council,  a beneficiary  will  be  entitled  to  a hearing  by  the 
Secretary  in  any  controversy  involving  more  than  SI 00  and  to  judicial  review  of 
the  Secretary's  decision  if  the  controversy  involves  more  than  SI. 000. 
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After  notification  of  ."i  adverse  decisi.in  ! y the  PSI'.O.  a pract 1 1 i< *<i  r ur 
provider  will  have  .in  opport  u tlv  for  fin  titer  discussion  .md  review  h\  ilie  I’SPO 
The  final  recommendations  H the  PSKO  are  referred  i<)  lire  Slale  Council  ami 
then  to  the  Secretary  who  mas  apply  atuhoti/cd  saneti  >n>.  Any  provider  who  is 
dissatisfied  with  an  adverse  decision  made  by  the  Secretary,  involving  exclusion 
from  the  Program  or  requiring  repayment,  is  entitled  to  a hearing  by  the  .'secretary 
and  to  judicial  review  of  the  Secretary's  final  decision. 

State  Councils 


In  any  State  with  three  or  more  PSKOV  the  Sei.tet.iry  will  establish  and 
appoint  the  membership  of  a State  Count. il  which  will  inelnd  : 

• One  representative  designated  by  each  PSRO 

• Pour  physicians  (two  designated  Ivy  the  State  medical  society  and  two 
designated  by  file  State  hospital  association). 

• P'our  knowledgeable  public  representatives  (two  iccommendc  I by  the 

Governor  of  the  State).  \ 

P'aclt  State  Council,  ami  the  PSKO's  in  stales  without  a Council,  wil  luve  an 
Advisory  Group  of  seven  to  eleven  membeis  including  repiesentatives  <«f  health 
vate  praetitioners,  otliei  i ! : an  physicians,  and  hospitals  and  other  faciliti  s which 
provide  Medicare  and  Medicaid  services  in  the  Stale.  The  Secretary  will  provide 
regulations  to  determine  the  manner  in  which  the  Advisoiy  Group  is  selected. 

The  State  Councils,  where  established,  will: 

• Review,  comment  upon,  and  transmit  to  the  Secretary  any  reports, 
received  from  the  PSKO's  concerning  violations  of  the  Program. 

• Review  any  appeals  to  a P§RO  decision  involving  more  than  SI 00. 

• Coordinate  the  activities  of,  and  disseminate  information  and  data 
among,  the  various  PSKO's  in  the  State. 

• Assist  the  Secretary  in  developing  uniform  data  gathering  procedures  io 
insure  efficiency  and  objective  compaiison. 

• Assisi  the  Secretary  in  the  evaluation  of  cash  PSKO 

• Assist  the  Secretary  in  developing  and  arranging  for  a qualified  PSRO 
replacement,  when  necessary. 
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National  Council 

A S itional  l*roh  Nion.it  ‘•’i.iiut  «i.l-  I!  \iov  ( o imil  of  eleven  physicians  will 
Ik  appointed  by  the  Seci.t.ny  lot  tlnee  year  i-nu.  with  elnhility  lor 
reappointment  Hu-  Sc.ivt.ir>  wll  dcsipi.ite  oik  ol  the  members  to  sene  ;ls 
Chairman  A majority  of  the  Council  will  he  nvninmendeil  hv  national 
organizations  representing  physicians.  but  the  Council  will  include  p!i> sui.ins 
recommended  by  consumer  groups  ami  other  health  care  interests.  All  members 
will  be  physicians  of  -ecngni'cd  standing  ami  distinction  in  the  appraisal  of 
medical  practice. 

The  National  I’lolessional  Standards  Review  Council  will: 


• Advise  and  assist  the  Secretary  in  the  administration  of  the  program. 

• Prepare  and  distribute  regional  ‘•norms."  with  periodic  revisions,  based 
on  a review  o‘  appi  >priafc  and  ideipiate  data. 

• Provide  technical  assistance  in  the  utilization  of  such  norm  . of  care  and 
treatment. 

• Review  tin  . utparalive  peifoimaiice  and  effect iveiv.s  of  State 
Council’s  an.l  1‘sRO's. 

• Reconinien.l  in.  asures  to  nnpi  nc  the  I’loei.un. 

• Submit  all  aiftial  report  to  the  Secretary  and  to  the  Cc.ngf.ss. 

Authorizations  to  "uc"  ,:i'on  the  PSHO 

T<»  the  c\t.  mi  m-ee.s.iry  or  appiopii.ite  for  the  proper  performance  of  its 
duties  and  functions,  the  I’SRO  serving  any  area  is  authorized,  in  accordance  with 
regulations  prc'  ii'vd  by  the  Secretary,  to 

(!)  Make  .ui.uigenients  to  utilize  the  services  of  piactitioners  or  specialists 
to  conduct  the  necessary  review. 

(2)  Undertake  professional  inquiry  be  tore  or  alter,  or  both  betore  and 
after,  the  provision  of  any  ser  iee. 

(.t)  lixamine  the  pertinent  records  o any  praetitioner  or  provider. 

(4)  Inspect  the  physical  facilities  in  which  care  is  rendered. 
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(5)  I'tiliv  ihe  s-ruces.  ard  tee.-pl  '!  ■ firdir-’s  of  j'.'a  hospital  review 
committees  v.  iih  dvmni'sii.uod  v '»r  •unco 

Hh  Utilize  Hu-  ' i%iws  i>l  tnedi.al  -o.  ictus  and  similar  organizations  to 
assist  tit  oil1.'  or  more  ot  the  review  activities  wlu'ii  lli>  y have 
demonstrated  then  capacity  to  dice  lively  peifoim  these  functions  in  a 
timely  fashion. 

AUcr  proper  notification.  no  M ulu  mo  or  Medicaid  claun  "ill  he  Paid  it  'he 
service  or  item  in  A|ueslion  is  sulked  to  icview  l>y  a I’sKO.  and  las  Ken 
disappiovcd  by  the  I’SIU). 

The  legislation  specifically  obligates  each  health  i.rre  prjcfilic  ner  and 
provider,  to  the  extent  of  their  intluencc  or  control,  to  pros  iJe  services. 

• Only  when  and  to  the  extent  that  they  are  medically  necessary 

v 

• Of  a quality  winch  meets  protessinually  reeogni/ed  standards  of  health 
care,  and 


#■  At  the  most  economic  level  of  service. 

l ailure  U»  comply  with  these  oblig.it ions,  in  a substantial  number  o!  cases,  or 
one  or  more  gioss  and  flagrant  violations  of  these  obligations  may  lead  to: 

• I'A.Iusioti  trom  el  eihihty  lor  leinilnir-ement  under  Medicare  and 
Medicaid  on  a permanent  basis  or  lor  a staled  peiiod  of  time.  >r 

• A requirement  foi  repayment  of  the  improper  service  or  $5,000 
whichever  is  less. 

Both  sanctions  would  he  applied  only  alter  pu  pel  application  of  he  appeals 
mechanism,  described  earlier,  and  exclusion  from  eligibility  for  payment  would 
become  effective  only  alter  notitiv  ilion  .o  the  pnbliv. 

The  provisions  of  ibis  Amendment  apply  to  the  operation  ol  any  State  plan 
approved  under  the  Social  Security  Act  as  a Medicare  or  Mcduaid  heuilh  care 

program. 

All  data  and  information  acquired  by  a I'SKO  is  confidential  and  v ill  only  be 
disclosed  on  a need  to  know  basis  or  as  piovulcd  by  tin-  Secretary  in  regulations 
Illegal  disclosure  may  rcsull  in  a line  "t  not  moic  than  M.nlUl  ar.d  imnnsonmviit 
of  not  more  than  sis  months,  or  both. 
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Limitations  on  I. ability  ;>i  • pto\  > ' 1 by  ilic  K'pisl  '.inn  lor  activities  in 
relation  lo  a l’SRO.  Notwithstanding  any  alicr  provision  ol  law 

• No  person  will  tv  liable  tor  • | invasion  of  relevant  information  which 
he  believes  lo  be  line.  In  a l”v:’(  i 

• In  the  ..  n. e ol  malice,  no  i ■ n is  liable  for  the  perfoi mar.ee  of  his 
duties  as  ; emplos  ee  or  aivn!  of  a I’SKO 

• No  prae  liln  ner  or  piosuler  is  liable  for  aitions  in  his  normal  work, 
taken  with  ih.c  ears',  in  conipb.m,  with  the  acce  pted  norni'  ol  care  and 
treatment  appli  el  b\  a I’Xl’.l  l 111  Ins  at  a 

Implenv  illation  of  the  Pu.'jrnm 

III  S eetetars  will  establish  .ipprojit i ■ te  aicas  • t I’SRO's.  Ihruujhout  tlie 

United  'states,  not  lalet  than  I mu.  re  I l'*74  .1 : 1 v 1 -I.  ■ ■ .ale  e|ualiltes]  >r ji3 nitra- 

tions as  I’SKO's  at  the  earliest  piaeti. le  elate  the real lei 

I’e ikIi np  assumption  ot  lespons:1  ■ ■ ! , t y ami  ele  u iistration  ol  capacity  for 
intprovc  ’ lit.  urrent  activities  will  • continued 

Hi  Secret.  r>  will  review  a 1 approve  a fonn.il  plan  foi  progressive 
issumpi  1 of  t'il!  ie*L ponsibihts  • • . ■ : itially  dea ••  ate  an  organization  'S  a I’SKO 
on  .a  c*  litiinai  I isis.  Durii.e  th.  tmil  pciiod  (not  to  exceed  74  months)  the 

Secieta  may  <\  me  the  i’SI’O  ' • pertoim  only  ‘united  duties  ami  functions. 

Assn  in  | 11  ol  iiorsibilitv  I « >1  1 lies  should  pro  ee»l  in  a.  ord  with  Ills' 
approvi  I | l.m  ■ that  at  the  end  ol  the  tual  penod.  the  I’SKO  is  performing  all 
reunites  .Kities  a d functions 

i fi  iditional  I'.sKO  agreements  may  be  terminated  by  either  party  on  90 

i days'  n>  tie e. 

Any  duties  and  functions  not  pcrfi'rmcd  by  a I’SKO  during  a trial  period, 
| , • will  continue  to  be  performed  as  presently  authorized  I lie  Secretary  is 

autho’i  ed  to  waive  any  other  review  requirements  when  substantial  evidence 

* indicate  s tliat  tl.c  I’SKO  meets  01  exceeds  those  requirements. 

; 

| Ti  e Secretary  will  provide  regulations  fot  correlation  and  cooperation 

between  c.riicrs.  intermediaries,  cow  rnment  agencies  and  PSRO’s  which  include 

• die  usage  ot  existing  data  gathering  capacity  w here  appropriate. 
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Dependent  Variable:  Length  of  stay 
Independent  Variable:  Secondary  diagnosis 


Level  of 

Secondary 

Diagnosis 

Median  Length 
of  Stay 

Average  Value  of 
Absolute  Deviatior, 
from  the  Median 

Number  of 
Observations 
in  Each  Cell 

0 

5.0  days 

1.5 

678 

1 

7.0  days 

3.3 

73 

2 

7.5  days 

4.3 

18 

3 

10.0  days 

2.6 

5 

4 

2.9.0  days 

37.0 

3 

5 

87.0  days 

0.0 

1 

r 

Exhibit  3b 

Cross  Tab i ■ la tlon  Using  Medians 

* 

Dependent  Variable:  Length  of  stay 
Independent  Variable:  Secondary  surgery 


Level  of 
Secondary 
Surgery 

Median  Length 
of  Stay 

Average  Value  of 
Absolute  Deviations 
from  the  Median 

Number  of 
Observations 
in  Each  Cell 

0 

5.0  days 

1.8 

740 

1 

7.0  days 

3.1 

25 

2 

8.0  days 

13.1 

10 

3 

17.0  days 

26.0 

3 

Exhibit  3c 


Cross  Tabulation  u sing  Medians 


Dependent  Variable:  Length  of  stay 
Independent  Variable:  Sex 


Average  Value  of 

K...nber  of 

Median  Length 

Absolute  Deviations 

Observations 

Sex 

of  Stay 

from  the  Median 

in  Each  Cell 

Male 

5.0  days 

1.8 

414 

Exhibit  3d 


r 


Cross  Tahi 1 1 a t i on  _U sing  Med  inns 


Dependent  Variable: 
Independent  Variable: 


Length  of  stay 
Age 


Age  Group 


Median  Length 
of  Stay 


Average  Value  of 
Absolute  Deviations 
from  the  Median 


Number  of 
Observations 
in  Each  Cell 


65  years 
or  less 


5.0  days 


1.7 


753 


Greater  than 
65  years 


12.0  days 


12.0 


25 


M 


mm 


Exhibit  3e 

Cross  Tabulation  Using  Medians 


Dependent  Variable: 
Independent  Variables: 


Length  of  stay 
Secondary  diagnosis 
& Secondary  surgery 


(a)  Midian  Length  of  Stay: 


levels 

of  Secondary 
iaeno sis 


Levels  of 
Secondary 
— ^gurgery 


5.0  days 

6.5  " 

9.0  " 

7.5  " 

8.0  " 


7.0  days 
6.5  " 

7.0  " 


2 

3 

6.0  days 

9.0  " 

13.0  days 

6.5  " 

- 

10.0  " 

- 

119.0  " 

- 

- 

87.0  " 

(b)  Average  value  of  absolute  deviations  from  the  median  length  of  stay 


Levels 

of  Secondary 
iaenosis 


Levels  of 
Secondary 
-^gurgery 


Number  of  observations  in  each  cell 


Levels  of 

Levels^ 

Secondary 

of  Secondary 

— ^Surgery 

Diagnosis 

^ 

Exhibit  3f 


Cross  Tabu lation  Using  Medians 

Dependent  Variable:  Length  of  stay 
Independent  Variables:  Secondary  Diagnosis, 

Secondary  Surgery, 
and  Age 


(a)  Median  length  of  stay: 


(i)  When  age  is  65  years  or  less 


(ii)  When  age  is  greater  than  65  years 


(b)  Average  value  of  absolute  deviations  from  the  median  length  of  stay: 


(1)  When  age  la  65  years  or  less 


(ii)  When  age  is  greater  than  65  years 
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Length  Of  Stay  During  No  Audit: 


OOTLIER5 


Exhibit  5b 


AUDITANT- 

-ICIfATION 


Exhibit  7 


SO  .0 


40.0 


BOX  PLOT 


30.0 


. *i(V 


20  0 


10.0 


$■; 


NOAUDIT  AUDIT  ANT  ACTUALAU 


Exhibit  8 


Surgeon 
Code  No. 


Analysis  by  Surf, cons  for  Primary  Cholecystectomy 


Number  of  Uncompli-  Year  of  Month  and  Year 

cated  Cases  Treated  Graduation  From  in  Which  Hospital 
1968- 1977 Medical  School  Privilege  Granted 


Age  of  the 
Surgeon 
(as  of  12/77) 


1 

38 

1935 

May  1946 

70  years 

2 

27 

1949 

Sept.  1958 

52 

3 

23 

1945 

May  1951 

54 

4 

21 

1930 

1935 

72 

5 

22 

1944 

Aptil  1950 

61 

6 

£6 

1956 

July  1965 

46 

7 

23 

1958 

July  1963 

47 

8 

34 

1962 

March  1967 

42 

9 

22 

1961 

October  1969 

43 

Exhibit  9 


Analysis  by  Surgeons  for  Acute  Appendicitis 


rgeon 
ie  No. 

Number  of  Uncompli- 
cated Cases  Treated 
1968-1977 

Year  of 

Graduation  From 
Medical  School 

Month  and  Year 
in  Which  Hospital 
Privilege  Granted 

Age  of  the 
Surgeon 
(as  of  12/77) 

1 

10 

65 

1950 

April  1957 

54  years 

11 

32 

1948 

May  1955 

55 

2 

20 

1949 

Sept.  1958 

52 

3 

41 

1945 

May  1951 

54 

4 

31 

1930 

1935 

72 

12 

55 

1944 

June  1955 

58 

6 

80 

1956 

July  1965 

46 

13 

66 

1958 

Nov.  1963 

45 

8 

56 

1962 

March  1967 

42 

9 

02 

1961 

Oct.  1969 

43 

Exhibit  10 


Surgeon 
Code  No. 


] 


2 


3 


4 


5 


6 


7 


6 


9 


Analysis  by  SurReon:  Primary  Cholecystectomy 

Number  of  Median  Length 

Time  Period  Patients  Treated  of  Stay 

Average  V tlue  of 
Absolute  Deviation 
From  t he  Median 

No  Audit 

22 

11  days 

1.95 

Audit  Anticipation 

9 

9 

1.44 

Actual  Audit 

7 

9 

0.71 

No  Audit 

12 

12 

2.83 

Audit  Anticipation 

11 

9 

1.27 

Actual  Audit 

4 

8 

1.75 

No  Audit 

12 

9 

2.03 

Audit  Anticipation 

5 

9 

0.40 

Actual  Audit 

6 

8 

1.17 

No  Audit 

18 

10 

2.39 

Audit  Anticipation 

3 

11 

3.00 

Actual  Audit 

- 

- 

- 

No  Audit 

10 

13 

1.70 

Audit  Anticipation 

4 

7.5 

2.00 

Actual  Audit 

8 

9.5 

2.00 

No  Audit 

36 

9 

2.28 

Audit  Anticipation 

26 

8 

2.08 

Actual  Audit 

24 

7 

1.21 

No  Audit 

5 

9 

2.20 

Audit  Anticipation 

13 

9 

2.31 

Actual  Audit 

5 

9 

3.40 

No  Audit 

8 

13 

3.12 

Audit  Anticipation 

10 

9 

1.60 

Actual  Audit 

16 

8 

1.25 

No  Audit 

3 . 

11 

2.67 

Audit  Anticipation 

7 

9 

1.14 

Actual  Audit 

12 

8 

0.67 

Doctors' 


U 


Exhibit  11 


Analyses  by  Doc  to  rs : Acute  Append ic  it  i n 


Time  Period 


No  Audit 

Audit  Anticipation 
Actual  Audit 


No  Audit 

Audit  Anticipation 
Actual  Audit 


No  Audit 

Audit  Anticipation 
Actual  Audit 


No  Audit 

Audit  Ant icipation 
Actual  Audit 


No  Audit 

Audit  Anticipation 
Actual  Audit 


No  Audit 

Audit  Anticipation 
Actual  Audit 


No  Audit 

Audit  Anticipation 
Actual  Audit 


No  Audit 

Audit  Anticipation 
Actual  Audit 


No  Audit 

Audit  Anticipation 
Actual  Audit 


No  Audit 

Audit  Ant  icipation 
Actual  Audit 


Number  of  Median 

Patients  Length 

Treated  of  Sta 


Avg.  Value  of  Absolute- 
Deviations  from  the 
Median 


6.00  days 

5.00  " 

6.00  " 


7.00  days 

5.00  " 

5.00  " 


6.00  days 

5.00  " 

5.00  " 


5.00  days 

5.00  " 

4.00  " 


7.00  days 
7.00  " 

0 


7.00  days 

6.00  ” 

5.50  " 


6.00  days 

4.00  " 

3.50  " 


5.00  days 

5.00  " 

4.00  " 


6.00  days 
6.00  " 

4.00  " 
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